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Abstract: We describe a system that lets a designer interactively draw patterns of strokes in the 
picture plane, then guide the synthesis of similar patterns over new picture regions. Synthesis is 
based on an initial user-assisted analysis phase in which the system recognizes distinct types of 
strokes (hatching and stippling) and organizes them according to perceptual grouping criteria. The 
synthesized strokes are produced by combining properties (e.g., length, orientation, parallelism, 
proximity) of the stroke groups extracted from the input examples. We illustrate our technique with 
a drawing application that allows the control of attributes and scale-dependent reproduction of the 
synthesized patterns. 
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Hachurage et pointillage par Pexemple 

Resume : Ce rapport presente une methode permettant a un artiste de dessiner interactivement un 
motif 2D de hachures ou de points puis de guider la synthese d'un motif similaire. La synthese 
s'appuie sur une phase d' analyse assistee par l'utilisateur dans laquelle le systeme extrait et organise 
des points ou des hachures (segments) selon des criteres de regroupement perceptuel. La synthese 
est alors effectuee en combinant les proprietes (longueur, orientation, parallelisme, proximite) des 
elements extraits par 1' analyse. 



Mots-cles : Rendu expressif 
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1 Introduction 

1.1 Motivation 

An important challenge facing researchers in non-photorealistic rendering (NPR) is to develop 
hands-on tools that give artists direct control over the stylized rendering applied to drawings or 
3D scenes. An additional challenge is to augment direct control with a degree of automation, to 
relieve the artist of the burden of stylizing every element of complex scenes. This is especially 
true for scenes that incorporate significant repetition within the stylized elements. While many 
methods have been developed to achieve such automation algorithmically outside of NPR (e.g., 
procedural textures), these kind of techniques are not appropriate for many NPR styles where the 
stylization, directly input by the artist, is not easily translated into an algorithmic representation. 
An important open problem in NPR research is thus to develop methods to analyze and synthesize 
artists' interactive input. 

In this work, we focus on the synthesis of stroke patterns that represent tone and/or texture. This 
particular class of drawing p rimitives have been investigated in the past (e.g., BSABS941 WS94, 
Ost99, DHvOSOO, DQ M + 0l1 ). but with the goal of accurately representing tone and/or texture 
coming from a photograph or a drawing. Instead, we orient our research towards the faithful 
reproduction of the expressiveness, or style, of an example drawn by the user, and to this end analyze 
the most common stroke patterns found in illustration, comics or traditional animation: hatching and 
stippling patterns. 

Our goal is thus to synthesize stroke patterns that "look like" an example pattern input by the 
artist, and since the only available evaluation method of such a process is visual inspection, 
we need to give some insights into the perceptual phenomena arising from the observation of a 
hatching or stippling pattern. In the early 20th century, Gestalt psychologists came up with a 
theory of how the human visual system structures pictorial information. They showed that the 
visual system first extracts atomic elements (e.g., lines, points, and curves), and then structures 
them according to various perceptual grouping criteria like proximity, parallelism, continuation, 
symmetry, similarity of color, velocity, etc. This body of research has grown consequently under the 
name of perceptual organization (see for example the proceedings of POCV, the IEEE Workshop 
on Perceptual Organization in Computer Vision). We believe it is of particular importance when 
studying artists' inputs. 

1.2 Related work 

The idea of synthesizing textures, both for 2D images and 3D surfaces, has been extensively 
addressed in recent years (e.g. by Efros and Leung [EL99], Turk [TurOl], and Wei and Levoy 
BWL011 ). Note, however, that this body of research is concerned with painting and synthesizing 
textures that are represented as images. In contrast, we are concerned with direct painting and 
synthesis of stroke patterns represented in vector form. I.e., the stroke geometry is represented 
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explicitly as connected vertices with attributes such as width and color. While this vector 
representation is typically less efficient to render, it has the important advantage that strokes can 
be controlled procedurally to adapt to changes in the depicted regions (strokes can vary in opacity, 
thickness and/or density to depict an underlying tone.) 

Stroke pattern synthesis systems have been studied in the past, for example to generate stipple 
drawings HDHvQSOOII . pen and ink representations [SABS9T1 [WS94IL engravings HOst99lh or for 
painterly rendering [Her98 |. However, they have relied primarily on generative rules, either chosen 
by the authors or borrowed from traditional drawing techniques. We are more interested in analysing 
reference patterns drawn by the user and synthesizing new ones with similar perceptual properties. 

Kalnins et al described an algorithm for synthesizing stroke "offsets" (deviations 

from an underlying smooth path) to generate new strokes with a similar appearance to those in 
a given example set. Hertzmann et al [HOCS02J, as well as Freeman et al [FTP03 ] address a 
similar problem. Neither method reproduces the inter-relation of strokes within a pattern. Jodoin 
et al | JEGPO02] focus on synthesizing hatching strokes, which is a relatively simple case in which 
strokes are arranged in a linear order along a path. The more general problem of reproducing 
organized patterns of strokes has remained an open problem. 

1.3 Overview 

In this paper, we present a new approach to analyze and synthesize hatching and stippling patterns 
in ID and 2D. Our method relies on user-assisted analysis and synthesis techniques that can be 
governed by different behaviors. In every case, we maintain low-level perceptual properties between 
the reference and synthesized patterns and provide algorithms that execute at interactive rates to 
allow the user to intuitively guide the synthesis process. 

The rest of the paper is organized as follows. We describe the analysis phase in Section [2 and the 
synthesis algorithm and associated "behaviors" in Section [3] We present results in Section ffl and 
conclude in Section [5] with a discussion of our method and possible future directions. 

2 Analysis 

We structure a stroke pattern according to perceptual organization principles: a pattern is a collection 
of groups (hatching or stippling); a group is a distribution of elements (points or lines); and an 
element is a cluster of strokes. For instance, the user can draw a pattern like the one in Figure [T] 
which is composed of sketched line segments, sometimes with a single stroke, sometimes with 
multiple overlapping strokes; our system then clusters the strokes in line elements that hold specific 
properties; and finally structures the elements into a hatching group that holds its own properties. 
We restrict our analysis to homogeneous groups with an approximate uniform distribution of their 
elements: hatching groups are made only of lines, stippling groups made only of points. This 
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Figure 1: A simple example of the analysis process in ID: the strokes input by the user (top) are 
analyzed to extract elements (bottom, in light gray), that are further organized along a path (dashed 
polyline). 

approach could be extended to more complex elements, using the clustering technique of Barla 
etal HBTS05L 

As a general rule of thumb, we consider that involving the user in the analysis gives him or her more 
control over the final result, at the same time removing complex ambiguities. Thus, in our system, the 
user first specifies the high-level properties of the stroke pattern he is going to describe. He chooses 
a type of pattern (hatching or stippling); this determines the type of elements to be analyzed (lines 
for hatching, points for stippling). He then chooses a ID or 2D reference frame within which the 
elements will be placed. He finally sets the scale £ of the elements, measured in pixels: intuitively, £ 
represents the maximum diameter of analysed points, and the maximum thickness of analyzed lines. 

Once these parameters are set, the user draws strokes as polyline gestures. Depending on the group 
type, points or lines at the scale £ are extracted and structured: Then statistics about perceptual 
properties of the strokes are computed. This whole processus has an instant feedback, so that the user 
can vary £ and observe changes made to the analysis in real-time. We first describe how elements 
are extracted given a chosen £ and their analyzed properties; then we describe how those elements 
are structured into a group, and how perceptual measures that characterize this group are extracted. 

2.1 Element analysis 

The purpose of element analysis is to cluster a set of strokes drawn by the user into points or lines, 
depending on the chosen element type. To this end, we use a greedy algorithm that processes strokes 
in the drawing order, and tries to cluster them until no more clustering can be done. We first fit each 
input stroke to an element (point or line) at the scale £. Strokes that cannot be fit to an element 
are flaged invalid and will be ignored in the remaining steps of the analysis. Then, valid pairs of 
elements that can be perceived as a single element are clustered iteratively. The fitting and clustering 
of points and lines is illustrated in Figure [2 

For points, the fitting is performed by computing the center of gravity c of a stroke S and measuring 
its spread s p = 2msLX pe s \p — c\. If s p > £, then the stroke is flaged invalid because the circle of 
center c and diameter s p do not encloses S. The clustering of two points is made by computing the 
center of gravity c* of the points and measuring its spread s* . Similarly, if ^* > £, then the points 
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cannot be clustered. This allows the system to recognize any cluster of short strokes relative to the 
scale £, like point clusters, small circled shapes, crosses, etc. (See Section|4l) 

For lines, the fitting is performed by computing the virtual line l v of a stroke S and measuring its 
spread si = 2max(d#(S',/ v ),d#(/ v ,S')) where dn(X,Y) = max xG x (min^y \x — y\) is the Hausdorff 
distance between two sets of points. The virtual line can be computed by least- square fitting, but 
in practice we found that using the endpoint line is enough and faster. Then, if 57 > £, the stroke is 
flaged invalid because the line segment of axis l v and thickness si do not enclose S. The clustering 
of two lines is made by computing the virtual line /* of the lines and measuring its spread s^ . The 
virtual line can be computed by least-square fitting on the whole set of points; but we preferred to 
apply least- square fitting only on the two endpoints of each clustered line for efficiency reasons. 
Then, if > £, the lines cannot be clustered. This allows the system to recognize any set of strokes 
that resembles a line segment at the scale £. Examples including sketched lines, overlapping lines, 
and dashed lines are shown in Section [4] 

Once points or lines have been extracted, we can compute their properties: extent, position and 
orientation. The extent property represents the dimensions of the element: point size or line length 
and width. For point size, we use the spread of the element. For lines, we use the length of the virtual 
line and its spread (for width). Orientation represents the angle between a line and the reference 
frame main direction (the main axis for ID frames, the X-axis for the cartesian frame). It is always 
ignored for points. We add a special position property for ID reference frames: since they are 
synthesized in 2D (in the picture plane), ID patterns have a remaining degree of freedom that is 
represented by the position of elements perpendicular to the main axis. For all these properties, we 
compute statistics (a mean and a standard deviation) and boundary values (a min and a max); We 
also store the gesture input by the user and will refer to it as the shape of the element in the rest of 
the paper. 



>1 S p <£ 



■ |S,<£ 



Figure 2: Top left: A stroke is fit to a point. Bottom left: A pair of points is clustered into a new 
point. Top right: A stroke is fit to a line. Bottom right: A pair of lines is clustered into a new line. 



2.2 Group analysis 

A group is considered to be an approximately uniform distribution of elements within a reference 
frame. This means that while analyzing a reference group, we are not interested in the exact 
distribution of its elements: we consider a reference group as a small sample of a bigger, 
approximately uniform distribution of the same elements. Consequently, we first need to extract 
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a local structure that describes the neighborhood of each element; this local structure will then be 
reproduced more or less uniformly throughout the pattern during synthesis. 

To this end, we begin with the computation of a graph that structures the elements locally: in ID, 
we build a chain that orders strokes along the main axis; whereas in 2D, we compute a Delaunay 
triangulation. We only keep the edges that: (a) connect two valid elements and (b) connect an 
element to its nearest neighbor. We chose this because the synthesis algorithm (described in 
Section [3]) converges only when considering nearest neighbor edges. However, this decision is also 
justified from a perceptual point of view: basing our analysis on nearest neighbors emphasizes the 
proximity property of element pairs, which is known to be a fundamental perceptual organization 
criterion. 

For each edge of the resulting graph, we extract the following perceptual properties, taking 
inspiration from Etemadi et al. |ESM + 91 ]: proximity for points and lines; parallelism, overlapping 
and separation for lines only. 



Proximity is simply taken to be the euclidean distance between the centers of the 
two elements in pixels. We not only compute this measure for points, but also for 
lines in order to initialize our synthesis algorithm (see Section [3]) 

Let's note A the vector from one point to the other, then we have 



prox - 



(1) 



with prox E [0, +«>). 



To compute parallelism, we first find the accute angle made between 
the two lines. Since there is no apriori order on the line pair, we take 
the absolute value of this accute angle and normalize it between and 
1. 

Let's note the accute angle, then we compute parallelism using 




par - 



71 



with par E [0,1]. 



(2) 



Like Etemaldi et al |ESM + 91 ], we define overlapping relative to the 
bissector of the considered line pair. But we modify slightly their 
measure to meet our needs: We project the center of each line on the 
bissector and use them to define an overlapping vector A. 

Overlapping is computed using the following formula: 

2IIAH 



ov - 



-L* 




(3) 
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where L\ and V 2 are the lengths of the lines projected on the bissector. Note that with this definition, 
ov = means a perfect overlapping. 



Finally, separation represents the distance between two lines, this time 
in the direction perpendicular to their bissector. We project the center 
of each line on a line perpendicular to the bissector and use them to 
define a separation vector T. 

Separation is then computed with the following formula: 

sep=\\r\\ 
with sep G [0,+«>). 

We compute statistics (a mean and a standard deviation) and bounds (a min and a max) for each of 
these properties. 




3 Synthesis 

The purpose of the synthesis process is to create a new stroke pattern that has the same properties 
(for elements and groups) as the reference pattern. We first describe a general algorithm that is able 
to create a new pattern meeting this objective; then we show how to customize it through the use of 
synthesis behaviors. 



3.1 Algorithm 

Our synthesis algorithm can be summarized as follows: 

1. Build a graph where the edge lengths follow the proximity statistics; 

2. Synthesize an element at each graph node using element properties; 

3. Correct elements position and orientation using element pair properties. 

The first step is achieved using Lloyd relaxation f lLlo82l . This technique starts with a random 
distribution of points in ID or 2D. It then computes the Voronoi diagram of the set of points, 
and moves each point to the center of its Voronoi region. When applied iteratively, the algorithm 
converges to an even distribution of points. Deussen et al. IDHvOS OOl observed that the variance of 
nearest neighbor edge length decreases with each iteration. We use this to get a variance (in nearest 
neighbor edge length) that approximately matches that of the reference pattern. 

Consider the mean /I*, standard deviation <7*, and the ratio r* = cr*//i* of a given property in our 
reference pattern. We start with a random point set by distributing N = N re fS^ / s^ re f points, where 
N re f is the number of elements in the reference pattern, and srf re f and srf are the area of the reference 
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and target patterns, respectively. We then apply the Lloyd technique, computing /I, a and r = o / jl 
of the current distribution at each step until r < r* . Note that jl will have changed throughout the 
set of iterations. Thus, in order to have jl = /i*, we finally rescale the distribution by An 
example of Lloyd's method is shown in Figure[3] 

In the second step, for each node of the graph we first pick a reference element E. Then we choose 
a set of element properties and compute a position, orientation and scale for E. There are many 
different approaches to choose element properties; the ones we implemented are detailed in the next 
section, and for now we only present the general algorithm. 



(a) (b) 

Figure 3: (a) An input random distribution and its Voronoi diagram, (b) The result after iteratively 
applying Lloyd's method until a desired variance-to-mean ratio in edge length is obtained. 

We first position the center of E at its corresponding node location. In the case of a ID reference 
frame, we also move E perpendicularly to the main axis using the relative position property. Then, 
E is scaled using the extent property; however, we impose a constraint on scaling for each type of 
element. In order for points to remain points, we ensure that their size is smaller than £; and similarly 
for lines, we ensure that their width is no more than £. Finally, E is rotated based on orientation. 
For a ID reference frame, we rotate E so that the angle with the local X-axis matches the orientation 
property. For a 2D reference frame, we use the angle with the global X-axis instead. 

Finally, in the third step, for each node of the graph, we compute a corrected set of parameters that 
takes into account the perceptual properties of nearest neighbor pairs extracted from the reference 
pattern during analysis. We use a greedy algorithm where each node is corrected toward its nearest 
neighbor in turn. In order to get a consistent correction, we add two procedures to this algorithm: 
first, the nodes are sorted according to the proximity with their nearest neighbor in a preprocess, so 
that the perceptually closest elements are corrected in priority; second, when a node is corrected, 
we discard both nodes of its edge from upcoming corrections, in order to ensure that the current 
correction stays valid throughout the algorithm. 

We now describe how an element is corrected based on perceptual measures. In a way similar to 
what we did for element properties, we choose a set of perceptual properties for element pairs. The 
details of how we perform this choice are explained in the next section. Note that the correction is 
not directly applied to the initial set of parameters: the user can control through linear interpolation 
the amount of correction he or she wishes to apply. 
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(a) 




Figure 4: (a) A point is corrected by displacing it along the direction to its nearest neighbor to 
match a proximity measure, (b) A line is first rotated to match parallelism, then displaced along and 
perpendicularly to the bissector direction to match overlapping and separation. 

For points, the only parameter to correct is position: we simply move the selected point along the 
line through its nearest neighbor to match the desired proximity (see Figure S^a).) Let's consider 
prox\ and prox^, the current and desired proximity values respectively. Then the correction applied 
to the position of current point is given by the following translation vector: 



If we are dealing with lines, we first correct the orientation of the current element based on 
the parallelism property we want to enforce; then we correct its position using overlapping and 
separation (see Figure Ua).) The reason why we first correct the orientation is that the overlapping 
property is highly dependent on the parallelism of lines. 

Let's consider par\ and par^, the current and desired parallelism values respectivelly. Then the 
correction applied to the orientation of current point is given by the following angle: 



Finally, we correct lines using a combination of overlapping and separation. For two overlapping 
values ov\ and ov^ for the current and target position, we translate the current line along the bissector 
line using the following vector: 




(5) 



1 pari^par 2 




(6) 



Ai 



.{0V2 — OV1). 




(7) 



HAill 



2 



Then we translate it perpendicularly to the bissector using: 



-n 



.{sep2-sepx) 



(8) 



lirill 



Note that the last two operations do not change the parallelism property. 
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3.2 Behaviors 

We now present the synthesis behaviors that are responsible for assigning a value for each property. 
We developed several behaviors because we believe that the ability to synthesize patterns that are 
more or less close to the reference pattern is a desirable feature: it lets us balance fidelity and 
variation relative to the reference pattern. 

We thus implemented three behaviors: sampling, copying and cloning, that range from close to the 
statistical distribution to close to the reference data. In the same spirit, we let the user choose the 
amount of correction that is applied. The correction results are displayed interactively. We now turn 
to the description of the three behaviors. 

Sampling This behavior produces patterns whose properties exhibit the same statistics as those 
of the reference pattern. For each property, we compute the mean and standard deviation of values 
in the input pattern to derive a Gaussian distribution function, then sample its inverse cumulative 
function to yield values that follow the distribution of the reference pattern. If the sampled value 
lies outside the range of values in the reference pattern, the sampling is repeated until a value in 
the original range of values is produced. The reference element whose shape is to be copied is then 
randomly chosen. 

Copying Moving toward increased fidelity to the reference pattern, this behavior assigns each 
property independently by copying values from randomly chosen elements in the reference pattern. 
For pairs, the reference pair with most similar value is found, and the synthesized pair is altered 
to match the reference pair. As an example, consider the proximity property of element pairs. If a 
nearest-neighbor pair of synthesized elements is separated by n pixels, we first find the reference pair 
whose proximity m is closest to n. We then correct the position of the chosen synthesized element 
to achieve a proximity of m pixels. For element shape, we first pick the reference pair with the most 
similiar proximity value and copy one of its element randomly. 

Cloning The cloning behavior synthesizes patterns that most closely follow the reference pattern. 
It is a modification of the copy behavior where all the properties are taken from the same source. 
Given a synthesized element, we randomly choose a reference element and copy all its properties 
to the synthesized element. Pairs are handled similarly, but the choice is not random: during 
correction, in order to stay close to the statistical distribution produced by Lloyd's method, we first 
find the reference pair with the most similar proximity property, then adjust parameters of the chosen 
synthesized element to yield the same pair- wise properties. The element shape is chosen like with 
the copying behavior. 
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4 Results 

Figure Oa)-(d) shows reference and synthesized hatching and stippling groups in ID and 2D. For 
these examples, we used the copying behavior and chose the correction amount manually. Note 
that complex elements extracted in the analysis phase are reproduced during synthesis: crosses 
and small circles for stippling, sketched strokes and multiple overlapping lines for hatching. The 
relations among nearest-neighbor synthesized elements are replicated from corresponding reference 
elements. FigureOe) shows a limitation of our method: the synthesis fails to reproduce recognizable 
stroke sequences seen in the input pattern. This is due to the small neighborhood size (only nearest 
neighbor) used in synthesis. The computation times are interactive, up to a couple of seconds for the 
most complex patterns we synthesized. 

We compare our different behaviors in Figure [3f)-(g), using two reference ID hatching patterns. 
With a quasi-uniform pattern (similar orientation, spacing, etc.), the sampling behavior has the 
advantage of synthesizing patterns with more variation, creating strokes in positions and orientations 
that were not present in the reference pattern, whereas the cloning behavior reproduces strokes and 
nearest neightbor relations found in the reference pattern. With a more irregular reference pattern, 
the sampling behavior produces patterns that lack the broader coherence of the example pattern, 
while the cloning behavior synthesizes patterns with more fidelity. Although not shown here, the 
copying behavior produces intermediate results, providing a trade-off between fidelity and variation. 

To illustrate our synthesis technique, we developed a 2D drawing application that lets the user draw 
example hatching or stippling patterns, then guide the synthesis of similar patterns over selected 
image regions. The system can vary stroke attributes such as color, thickness and opacity according 
to colors or tones in a provided background image (see Figure left). This lets us create colored 
strokes that represent shadows, highlights and intermediate tones. The final illustration is composed 
of the synthesized patterns, optionally composited on top of the background image. The system can 
also synthesize multiple versions of the same pattern at different resolutions, supporting the scale- 
dependent reproductions of the output image, for simple levels-of-detail or for printing purpose (see 
Figure right). 



5 Discussion and future work 

The main limitation of our approach is that we only consider nearest neighbor relations. To our 
knowledge, the first and only attempt to deal with larger neighborhoods is the work of Jodoin 
et al |JEGPO02]. However, they only consider parametric elements of the same dimension, 
organized along a ID path with no perceptual analysis (though they acknowledge the importance 
of extracting low-level perceptual properties). On the other hand, our approach is a step in another 
direction: we consider more general elements thanks to our element analysis, and introduce low- 
level perceptual properties in our group analysis. We also generalize the synthesis to 2D patterns. 
These two approaches are not incompatible, however. We look forward to combining advantages 
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of both methods to develop a more general solution to the example-based stroke pattern synthesis 
problem. 

We believe that extending our method to bigger neighborhoods, exploiting perceptual properties 
for neighborhood comparisons, is the logical next step towards this solution. It is interesting to 
compare our method to work done on texture synthesis on surfaces (e.g., Turk HTurOU and Wei 
and Levoy [WL01 ]). In those systems and ours, the first step is to build a lattice that models the 
correspondence between the input (2D texture or reference stroke pattern) and the output (a 2D 
surface or target distribution of elements). The second step initializes nodes of the lattice with 
values that follow the statistics of the reference pattern or texture. In the final step, node values 
are modified based on neighborhood comparisons of various sizes, either at random BWL01I or in a 
predefined order (line sweeping in [TurOl ], nearest neighbors in our work). This observation opens 
promising avenues for building on the existing body of texture synthesis techniques. 

For our clustering algorithm, we relied on the drawing order, but we might investigate other 
orderings, for instance based on proximity. This would even be mandatory in cases where the 
reference pattern is extracted from an image and the drawing order is not known. We also considered 
only points and lines and the perceptual relations among them. However, other primitives like arcs 
or more complex curves have been studied from a perceptual organization point of view, and we 
plan to incorporate them in our system in future work. Other perceptual criteria like symmetry 
or closeness might then be of great value for those patterns. Finally, even if we believe that a 
user-assisted analysis is the most valuable approach, one might consider automating the process for 
specific applications (e.g. for capturing the style of an existing drawing). This implies determining 
the group type, reference frame and £ automatically. 



6 Conclusions 

We presented a new approach to stroke synthesis by example, for two particular classes of patterns: 
hatching and stippling (in ID and 2D). Our method is fast and easy to implement. Its interactive 
response and its different synthesis behaviors let the user guide the synthesis process. The resulting 
synthesized patterns are perceptually similar to the reference ones, but also add a degree of variation. 
This lets us use our tool in a drawing application, with features such as stroke attribute control for 
efficient image depiction, and scale-dependent synthesis for levels-of-detail. 
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Figure 5: (a)-(d) Stippling and hatching results in ID and 2D with their reference pattern (small 
boxes); (e) A synthesis failure example; (f) For uniform patterns, sampling (top) introduces more 
variation than cloning (bottom); (g) For less uniform patterns, sampling (top) is more incoherent 
than cloning (bottom). 
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Figure 6: Two examples from our drawing application. Left: the thickness of the lines is controled 
by a user-defined mask; Right: levels-of-detail of a drawing (bottom) are automatically synthesized 
from an initial resolution (top). 
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